Method for encoding signals, related systems and program product therefor

ABSTRACT

A method for encoding video signals subjects the signals to unbalanced multiple description coding. The unbalanced multiple description coding codes a video signal in a first high resolution packet and a second low resolution packet and represents, respectively a first high resolution description and a second low resolution description. The unbalanced multiple description coding step includes using different intra refresh periods for the first and second high resolution descriptions, with an intra refresh period for the second low resolution description shorter than the intra refresh period of the first high resolution description.

RELATED APPLICATION

The present application is a continuation of U.S. patent applicationSer. No. 11/215,313 filed on Aug. 29, 2005. The disclosure of which isherein specifically incorporated by this reference in its entirety.

FIELD OF THE INVENTION

The present invention relates to coding techniques, for instance forvideo signals. The present invention can be applied to anymulticast/broadcast video distribution scenario where packet loss mayoccur. The proposed technique is particularly suitable for videotransmission in Wireless Local Area Network (WLAN) hotspots and forvideo broadcasting in Third Generation (3G) cellular networks.

BACKGROUND OF THE INVENTION

Video communication over lossy packet networks such as the Internet orwireless links is hampered by packet loss. Indeed, video qualityseverely degrades in presence of lost packets.

Furthermore, video communication over lossy packet networks is hamperedby limited bandwidth and packet loss. In fact, video coders commonly usepredictive coding schemes in order to reduce temporal correlation, andthe main drawback of these schemes is that even a single packet loss maycause errors during the decoding process that propagate in time.

Intra coding may be used to limit the effect of errors, however the highbit rate required limits its use in many applications.

Another common way to increase the robustness of the coded stream and toreduce the length of error propagation is the use of the Forward ErrorCorrection (FEC) codes. This last solution provides robustness to packetloss at expenses of coding efficiency. The main limitation of ForwardError Correction codes is the “threshold effect”: when the number oflost packets exceeds the correction capability of the code (e.g. thenumber of redundant packets for Reed-Solomon codes), the code is unableto correct the errors, thus causing quality degradation.

Layered or scalable approaches essentially prioritize data and therebysupport intelligent discarding of the data (the enhancement data can belost or discarded while still maintaining usable video), however thevideo can be completely lost if there is an error in the base layer.

Multiple Description (MD) Coding attempts to overcome this problem bycoding a signal into multiple independent bit-streams such that anybit-stream can be used to decode a baseline signal. If one stream islost, the other streams can still be decoded to produce usable video,and most importantly, the correctly received streams enable improvedstate recovery of the corrupted stream. It is possible to useinformation from the multiple streams to perform state recovery at thedecoder. The main problem of Multiple Description is codinginefficiency. At the same rate, a single description produces higherquality than Multiple Description.

To overcome this limitation, some solutions based on Unbalanced MultipleDescriptions (UMD) have been investigated by researchers.

With unbalanced operation, the descriptions have different importanceduring the video reconstruction process. This way, it is simpler tocontrol the amount of redundant data to add.

To achieve unbalanced operation one can adapt the quantization, theframe rate and the spatial resolution.

With unbalanced operation, different descriptions have differentimportance during the video reconstruction process. Many differentsystems to generate the Unbalanced Descriptions (UD) have been analyzedin the related prior art. The existing techniques are based on:

-   -   particular transformations,    -   energy considerations,    -   frame rescaling operations, and    -   different quantization operations.

Many prior art solutions have been designed to send the descriptionsover different channels. Indeed, some existing solutions exploit pathdiversity either at the link layer using different antennas, or at theInternet Protocol (IP) level using multiple senders.

Introducing path diversity at the link layer can complicate the designof the video devices, requiring link-layer modifications to allowcross-layer optimizations. On the other side, having a plurality ofsenders can complicate the network topology.

Document WO-A-2004/046879 discloses an apparatus and method forgenerating multiple descriptions of compressed data. In the apparatusand method, transform coefficients are generated from input data andquantized. An energy distribution of the quantized transformcoefficients is generated. Based on the energy distribution, thetransform coefficients are grouped into layers. By entropy codingdifferent number of layers, multiple descriptions of compressed data aregenerated.

Document JP-2002/198821 discloses a method and a device for processingsignal for transmission in wireless communication system. Amultiple-description coder generates many different descriptions in aprescribed portion of signals in a wireless communication system byusing Multiple Description Scalar Quantization (MDSQ) or another type ofmultiple description coding. The different descriptions in theprescribed portion of the signals are arranged in a plurality ofpackets, so that at least the first description in the prescribedportion may be arranged in the first packet and the second descriptionmay be arranged in the second packet. Each packet is transmitted byusing a frequency hopping modulator and the hopping rate of themodulator is selected or constituted based, at least partially, on thenumber of descriptions generated with respect to the different portionsof the signals.

In U.S. Pat. No. 6,801,532 a process of sending packets of real-timeinformation at a sender includes the steps of initially generating atthe sender the packets of real-time information with a source rategreater than zero kilobits per second, and a time or path or combinedtime/path diversity rate, the amount of diversity initially being atleast zero kilobits per second. The process sends the packets, therebyresulting in a Quality of Service (QoS), and optionally obtains at thesender a measure of the Quality of Service. Rate/diversity adaptationdecision may be performed at receiver instead. Another step compares theQuality of Service with a threshold of acceptability, and when theQuality of Service is on an unacceptable side of said thresholdincreases the diversity rate and sends not only additional ones of thepackets of real-time information but also sends diversity packets at thediversity rate as increased.

In U.S. Pat. No. 6,754,203 a method for communicating data over a packetswitched network comprises dividing data into a plurality of frames,with each frame described by at least a first and a second parameter.The second parameter has a high correlation. The first parameter isplaced in a first and a second description, while the second parameteris interleaved to the first and second descriptions. The first andsecond descriptions are packetized and communicated over the network.Upon reception, the first parameters for a frame sequence are extractedfrom one of the packets, while the interleaved second parameters areextracted from both the packets. If a packet is lost, the missing firstparameter may be obtained from another packet, while the missing secondparameter may be reconstructed using a second parameter from the otherpacket.

In U.S. Pat. No. 6,757,735 a method and system for streaming media datato a fixed client and/or a mobile client are disclosed, providing forencoding media data to be streamed to a client into a first multipledescription bit-stream and into a second multiple descriptionbit-stream. The method then determines the appropriate plurality ofservers from a network of servers onto which the first and secondmultiple description bit-streams should be distributed. Then it isprovided for distributing the first and second multiple descriptionbit-streams to the appropriate plurality of servers positioned atintermediate nodes throughout a network such that a client is providedwith access to the media data via a plurality of transmission paths.

In U.S. Pat. No. 6,460,153 is described method based on a projectiononto convex sets (POCS) for consistent reconstruction of a signal from asubset of quantized coefficients received from an N.times.Kover-complete transform. By choosing a frame operator F to be theconcatenization of two or more K.times.K invertible transforms, the POCSprojections are calculated in R.sup.K space using only the K.times.Ktransforms and their inverses, rather than the larger R.sup.N spaceusing pseudo inverse transforms. Practical reconstructions are enabledbased on, for example, wavelet, sub-band, or lapped transforms of anentire image. In one embodiment, unequal error protection for multipledescription source coding is provided. In particular, given a bit-planerepresentation of the coefficients in an over-complete representation ofthe source, it is provided coding the most significant bits with thehighest redundancy and the least significant bits with the lowestredundancy. In one embodiment, this is accomplished by varying thequantization step-size for the different coefficients. Then, theavailable received quantized coefficients are decoded using a methodbased on alternating projections onto convex sets.

In U.S. Pat. No. 6,215,787 is disclosed a signal data processing methodusing equal importance packetization. Processing of image data and othertypes of signal data is provided by representing the signal data in sucha way that, when separated into packets, all packets are ofapproximately the same importance. As a result, if some of the packetsare, for example, randomly lost in a lossy packet network, the resultingdegradation in a reconstructed version of the signal is substantiallyuniform regardless of which packets are lost. A given image or othersignal may be separated into packets using an energy equalizationprocess, a signal whitening process, or other suitable technique.

The topics considered form the subject of extensive technicalliterature, as witnessed e.g. by the following technical papers:

-   -   “Unbalanced Quantized Multiple Description Video Transmission        using Path Diversity”, Sila Ekmekci, Thomas Sikora, Technical        University Berlin, in which it is disclosed an approach to        Multiple Description based on the Multiple State Video Coding        achieving a flexible unbalance rate of the two streams by        varying the quantization step size while keeping the original        frame rate constant. The total bit-rate RT is fixed which is to        be allocated between the two streams. If the assigned bit-rates        are not balanced there will be PSNR (peak signal to noise ratio)        variations between neighbouring frames after reconstruction. It        is attempted to find the optimal rate allocation while        maximizing the average reconstructed frame PSNR and minimizing        the PSNR variations given the total bit-rate RT and the packet        loss probabilities p1 and p2 over the two paths. The        reconstruction algorithm is also taken into account in the        optimization process. Results presenting optimal system designs        for balanced (equal packet loss probabilities) but also for        unbalanced path conditions (different packet loss probabilities)        are reported;    -   “Unbalanced Multiple Description Video Coding With        Rate-Distortion Optimization”, David Comas, Raghavendra Singh,        Antonio Ortega and Ferran Marques, deals with the problem of        robust streaming of video data over best effort packet networks,        such as the Internet, proposing multiple descriptions coding to        protect the transmitted data against packet losses and delay,        while also ensuring that the transmitted stream can be decoded        with a standard video decoder, such as the H.263 decoder. The        video data is encoded into a high resolution, i.e., high        quality, video stream (description) using an encoder that        produces an H.263 compliant stream. In addition, a        low-resolution video stream (description) is also generated by        duplicating the important information from the high-resolution        video stream. This information includes the headers, the motion        vectors and some of the discrete cosine transform (DCT)        coefficients of the high resolution video stream. The remaining        DCT coefficients are set to zero in the low-resolution video        stream. Hence both video streams are independently decodable by        a standard H.263 video decoder. However, only in case of a loss        in the high-resolution video stream, the corresponding        information from the low resolution video stream is decoded,        else the received high resolution video stream is decoded. Thus        the system is an example of an unbalanced MDC system where the        low-resolution description is used only in case of losses in the        high-resolution description. An optimization algorithm is        disclosed which, given the probability of packet loss, allocates        bits to the high resolution and low resolution descriptions, and        selects the right number of coefficients to duplicate in the low        resolution description, so as to minimize the expected        distortion. The article refers to MD video coder using a similar        rate allocation scheme however generating balanced descriptions;    -   “End-To-End Rate-Distortion Optimized Mode Selection For        Multiple Description Video Coding”, Brian A. Heng, John G.        Apostolopoulos, and Jae S. Lim, discloses use of Multiple        Description video coding to reduce the detrimental effects        caused by transmission over lossy packet networks. Each approach        to MD coding consists of a tradeoff between compression        efficiency and error resilience. How effectively each method        achieves this tradeoff depends on the network conditions as well        as on the characteristics of the video itself. This paper        proposes an adaptive MD coding approach, which adjusts to these        conditions through the use of adaptive MD mode selection. The        encoder selects between MD coding modes in a rate-distortion        optimized manner to most effectively trade-off compression        efficiency for error resilience;    -   “A Novel Error-Concealment Algorithm for an Unbalanced Multiple        Description Coding Architecture”, Marco Fumagalli, Rosa Lancini,        and Stefano Tubaro, discloses, in order to increase the        robustness of the coded stream and to reduce the length of error        propagation, the use of a Multiple Description Coding technique,        applying a novel sequence-based Error Concealment (EC) algorithm        to an unbalanced MD video coding system that generates a        High-Resolution (HR) and a Low-Resolution (LR) description. In        order to recover a loss in the current frames the EC algorithm        takes into account not only the spatial neighboring of the        region to which correspond the data loss, but it looks also at        what will happen in a significant number of future frames        looking both the HR and the LR descriptions;    -   “Unbalanced Multiple Description Video Communication Using Path        Diversity”, John G. Apostolopoulos and Susie J. Wee, combines MD        video coding with a path diversity transmission system for        packet networks such as the Internet, where different        descriptions are explicitly transmitted through different        network paths, to improve the effectiveness of MD coding over a        packet network by increasing the likelihood that the loss        probabilities for each description are independent. The        available bandwidth in each path may be similar or different,        resulting in the requirement of balanced or unbalanced        operation, where the bit rate of each description may differ        based on the available bandwidth along its path. The MD video        communication system disclosed is effective in both balanced and        unbalanced operation. Specifically, unbalanced MD streams are        created by carefully adjusting the frame rate of each        description, thereby achieving unbalanced rates of almost 2:1        while preserving MD effectiveness and error recovery capability.

From the foregoing description of the current situation, it emerges thatthere exists the need to define solutions capable of dealing withmulticast video transmissions in a more satisfactory way as compared tothe solutions according to the known art described previously.

SUMMARY OF THE INVENTION

The object of the invention is thus to provide a fully satisfactoryresponse to this need.

According to the present invention, that object is achieved by means ofa method having the features set forth in the claims that follow. Theinvention also relates to a corresponding system as well as a relatedcomputer program product, loadable in the memory of at least onecomputer and including software code portions for performing the stepsof the method of the invention when the product is run on a computer. Asused herein, reference to such a computer program product is intended tobe equivalent to reference to a computer-readable medium containinginstructions for controlling a computer system to coordinate theperformance of the method of the invention. Reference to “at least onecomputer” is evidently intended to highlight the possibility for thepresent invention to be implemented in a distributed/modular fashion.

The claims are an integral part of the disclosure of the inventionprovided herein.

The solution described herein generates two descriptions havingdifferent resolution and different quantization. The solution usesdifferent intra refresh periods (and thus different Group Of Pictures(GOP) structures) for the produced descriptions. In particular thesolution employs shorter intra refresh period for Low Resolution (LR)description (in order to improve its resilience to packet losses), andhigher intra refresh period for High Resolution (HR) description (inorder to improve coding efficiency). All prior art solutions based ondifferent resolutions and/or different quantization, instead, use thesame intra refresh period for all the descriptions.

Another difference of the solution described herein compared to existingtechniques is the packetization scheme. The latter keeps into accountthe network Maximum Transfer Unit (MTU) in order to achieve good networkefficiency.

Indeed the joint High Resolution and Low Resolution encoders exchangeinformation in order to produce Unbalanced Multiple Description (UMD)packets having size equal to the Maximum Transfer Unit. Thecommunication between High Resolution and Low Resolution encoders allowsthem to adjust the packet size and the quantization so that theaggregation of High Resolution and Low Resolution packets has size equalto the Maximum Transfer Unit. Moreover, this communication between theencoders allows the packetizer to keep constant the frame offset betweenHigh Resolution and Low Resolution packets aggregated in a singleUnbalanced Multiple Description packet. Without such an approach, theoffset could not be kept constant and the size of the aggregated packetwould be either smaller or bigger than Maximum Transfer Unit, causingnetwork inefficiency.

The packetization scheme described herein has been designed for use on asingle channel, without the need to change existing link layertechnology. The solution does not need a cross-layer approach, andtherefore is link-layer independent. Moreover, this packetization schemedoes not need to have multiple senders at IP layer and so it does notcomplicate network topology.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will now be described, by way of example only, withreference to the enclosed figures of drawing, wherein:

FIG. 1 shows an exemplary block diagram of a sender,

FIG. 2 shows an example of Unbalanced Multiple Description payloadformat,

FIG. 3 shows an exemplary block diagram of a receiver,

FIG. 4 shows the operating mode of a postprocessor according to thepresent invention,

FIG. 5 shows an example of video streaming in a WLAN scenario, and

FIG. 6 shows an example of video streaming in a 3G scenario.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION

The solution described herein illustrates a set of techniques based onUnbalanced Multiple Description Coding (UMD) to improve the robustnessof video transmissions.

The exemplary system described herein relies on two descriptions, a HR(High Resolution) description and LR (Low Resolution) description. TheHigh Resolution description has higher quality than the Low Resolutionone. The High Resolution and the Low Resolution descriptions areproduced by using different encoding parameters.

The solution described, compared to other techniques based on UnbalancedMultiple Description, makes use of different intra refresh periods (andthus different Group Of Pictures (GOP) structures) for the produceddescriptions. In particular the solution proposes to use shorter “intrarefresh” period for the Low Resolution description (in order to improveits resilience to packet losses), and higher “intra refresh” period forthe High Resolution description (in order to improve coding efficiency).A packetization scheme has been introduced in order to optimize networkperformance. The techniques described herein are able to produceacceptable quality even if redundancy percentage is less than packetloss rate.

In this scheme the Low-Resolution (LR) description is primarily used asredundancy, and it is employed to conceal errors (or losses) in theHigh-Resolution (HR) description, and vice-versa. The Low Resolutionstream in the Unbalanced Multiple Description case although has somesimilarity with the redundancy added by the code in the Forward ErrorCorrection approach, is different since the Forward Error Correction isa general technique, independent of video encoding technology, while theUnbalanced Multiple Description is able to exploit video properties.

The Low Resolution bandwidth is a small fraction of the High Resolutionone. Using the same packet size for both the High Resolution and the LowResolution, then the number of Low Resolution packets would be much lessthan the number of High Resolution packets.

Moreover, a single Low Resolution packet would include more macroblocksthan an High Resolution one: a single packet loss in the Low Resolutionstream corresponds to the loss of a greater number of packets in theHigh Resolution stream. This would reduce the error-recovery capabilityof such an approach. To overcome this problem, it is desirable that thetotal number of Low Resolution packets be approximately equal to thenumber of High Resolution packets. This means that the Low Resolutionpacket size must be smaller than the High Resolution one.

As most Multiple Description and Unbalanced Multiple Descriptiontechniques use the same “intra” period for all the descriptions, asalready mentioned, the solution described herein uses a lower “intra”period (more frequent “intra refresh”) for the Low Resolution streamcompared to the High Resolution one. Even if this solution reduces thecoding efficiency for the Low Resolution stream, it preserves theefficiency of the High Resolution stream. On the other side, it makesthe Low Resolution more robust to packet loss. Intra refresh can also becoordinated among descriptions to ensure that at least one descriptionhas been recently refreshed for every frame.

When the same frame is lost in both the High Resolution and in the LowResolution, error propagation can be stopped as soon as a Low Resolutionintra frame is received. The reconstructed video will be low quality,until a High Resolution intra frame is received. Even if Low Resolutionis low quality, it is intelligible.

The High Resolution and the Low Resolution encoders produce packetsready to be encapsulated in Realtime Transport Protocol (RTP). Bestresults can be achieved if packet losses of the two streams areuncorrelated. Virtual independent channels can be created by suitableinterleaving policy between High Resolution and Low Resolution packets.A possible packetization scheme would use two different RealtimeTransport Protocol sessions, one for the High Resolution and one for theLow Resolution streams.

The main problem of this approach is its network inefficiency, sincehalf of the packets would be very small. The solution described hereinovercomes this limitation by aggregating a High Resolution packet and aLow Resolution packet in a single User Datagram Protocol (UDP) packet.

The Unbalanced Multiple Description architecture is described byanalyzing both the sender (the encoder) and the receiver (the decoder)sides.

FIG. 1 represents the main blocks of a system for encoding videosignals, or sender, 10.

A video input 15 is fed to a H264 encoder 20, that produces a stream ofHigh Resolution packets, and to a H264 encoder 30, that produces astream of Low Resolution packets. Such two streams are fed to apacketizer block 40.

Both the encoders receive the same uncompressed video input 15. They usethe same codec but with different encoding parameters. The HighResolution and the Low Resolution encoders generate respectively the HR(High-Resolution) and LR (Low-Resolution) video streams. The HighResolution is the primary stream and the Low Resolution is the redundantone.

The output packets from the High Resolution encoder 20 and the LowResolution encoder 30 are taken as input by a packetizer 40. The latterblock 40 aggregates the High Resolution and the Low Resolution packetsinto a single Unbalanced Multiple Description packet 50.

The aggregation scheme operates by aggregating a Low Resolution packetbelonging to frame n and a High Resolution packet belonging to frame k(with k=n−Offset). The Offset value can be an integer number within agiven interval, bounded by a minimum and a maximum value. In the teststhe offset has been bounded between 30 and 35 values. Greater values forthe Offset can help achieving uncorrelation between the High Resolutionand the Low Resolution packet losses. On the other side, to a greaterOffset corresponds a greater delay and a greater buffering requisiteimposed on the receivers.

An example of Unbalanced Multiple Description payload format isrepresented in FIG. 2.

An Unbalanced Multiple Descriptions header 60 is 4 octets long. A HighResolution size field 62 (the most significant 2 octets) represents thelength of a High Resolution packet 70. If the Unbalanced MultipleDescriptions packet includes only a Low Resolution packet 80, then theHigh Resolution size field 62 is null.

A Low Resolution size field 64 (the remaining 2 octets) represents thelength of the Low Resolution packet 80. If the Unbalanced MultipleDescriptions packet includes only the High Resolution packet 70, thenthe Low Resolution size field 64 is null.

After the payload there is the High Resolution packet 70 (if available)followed by the Low Resolution packet 80 (if available). Both the HighResolution size field 62 and the Low Resolution size field 64 cannot benull at the same time.

A very small fraction of the Unbalanced Multiple Descriptions packetsmay include only a High Resolution packet or only a Low Resolution one.This may happen because the packetizer block 40 must keep approximatelyconstant the offset between the High Resolution and the Low Resolution,and because the maximum size of an aggregated Unbalanced MultipleDescriptions packet must not exceed the network Maximum Transfer Unit(MTU).

Indeed, exceeding the Maximum Transfer Unit would require IPfragmentation, so that each fragment does not exceed the MaximumTransfer Unit and can be sent on the network.

If a single fragment of an IP datagram were lost, all the correctlyreceived fragments belonging to that same datagram would need to bedeleted. This may cause severe network performance degradation,especially on wireless links and/or in congested networks. On the otherside, if the produced packets are much smaller than the Maximum TransferUnit, network is used inefficiently (excessive overhead).

Summarizing, the solution described herein poses the followingfundamental constraints in order to achieve robust and efficientoperation:

-   -   the Low Resolution bandwidth is a pre-defined small fraction of        the High Resolution bandwidth;    -   the High Resolution and the Low Resolution have a similar number        of total packets;    -   the Offset between the High Resolution and the Low Resolution        packets aggregated in a single Unbalanced Multiple Descriptions        packet is kept approximately constant;    -   the size of aggregated Unbalanced Multiple Descriptions packets        is kept as close as possible to the Maximum Transfer Unit,        without exceeding the Maximum Transfer Unit.

In order to satisfy the constraints, one can tune the quantization, theRealtime Transport Protocol packet size and other encoding parameters.If the High Resolution encoder 20 and the Low Resolution encoder 30 wereindependent, satisfying those constraints would be really complex andwould require a great number of consecutive trials. Thus, independentlyencoding the High Resolution and the Low Resolution streams, is not apractical solution.

The technique described herein proposes to use joint encoding betweenthe High Resolution and the Low Resolution encoders, so that consecutivetrials are not needed. The joint encoders exchange a great number ofstatistics about the status of the produced streams, and take immediateactions to satisfy the constraints.

For example, if the number of the High Resolution packets is too bigcompared to the Low Resolution ones, the High Resolution encoder 20 willincrease its current Realtime Transport Protocol packet size, while theLow Resolution encoder 30 will decrease its current one. Correctiveactions in the encoders may tune any combination of encoding parameterson the fly, thus guaranteeing that the four constraints stated above aresatisfied.

Moreover, the joint encoding process is computationally more efficientthan independent encoding because some operations can be executed onlyonce. For example, Motion Estimation may be performed only in the HighResolution encoder 20: the Low Resolution encoder 30 could use arescaled version of the High Resolution motion vectors.

The technique described herein uses joint encoding to improve thepacketization scheme and network efficiency.

As stated, most Unbalanced Multiple Descriptions techniques use the same“intra” period for all the descriptions. The technique described hereinuses a lower “intra” period for the Low Resolution stream compared tothe High Resolution one. In the tests 12 frames intra period has beenused for the High Resolution stream and only 3 frames intra period forthe Low Resolution stream. Even if this solution reduces the codingefficiency for the Low Resolution, it preserves the efficiency of theHigh Resolution. The advantage is that the Low Resolution stream becomesmore robust to packet loss.

When the same frame gets lost in both the High Resolution and the LowResolution streams, error propagation can be stopped as soon as a LowResolution intra frame is received. The reconstructed video will be lowquality, until a High Resolution intra frame is received. However, evenif the Low Resolution is low quality, it is intelligible.

FIG. 3 represents the main blocks of a receiver 90.

The receiver 90 decodes the received packets 50 of the UnbalancedMultiple Descriptions stream and performs an Error Concealment (ERC)operation.

The receiver block 90 comprises a de-packetizer block 100, a H264decoder 110 (that creates the High Resolution stream) and a H264 decoder120 (that creates the Low Resolution stream). The receiver block 90further comprises a downscaler block 130, an upscaler block 140 and apostprocessor block 150.

The de-packetizer block 100 extracts the High Resolution and the LowResolution packets from the Unbalanced Multiple Descriptions packet. Theextracted packets are sent to their respective decoders 110 and 120.After decoding, the Low Resolution frames are upscaled to HighResolution by the upscaler block 140. HR frame and the upscaled versionof LR frame are taken as input by the postprocessor block 150. Thelatter performs Error Concealment (ERC). If a given macro-block has beenlost in the High Resolution frame, it can be recovered from thecorresponding macro-block of the Low Resolution frame.

When performing Error Concealment operation, the postprocessor 150operates on frames from the High Resolution and the Low Resolution withthe same sequence number. The frame produced by the postprocessor 150 issent to the playback unit and is also sent back to the decoders. Theoutput frame is downscaled in block 130 before being accepted by the LowResolution decoder 120.

FIG. 4 describes how the postprocessor 150 operates. The postprocessor150 receives n-th frame from the HR decoder 110 and from the upscaler140.

The postprocessor 150 includes an Error Concealment (ERC) block 170 anda buffer 155. The High Resolution and the Low Resolution frames havingthe same sequence number are taken as input by the postprocessor 150.The latter performs error detection to identify corrupted macro-blocksin the decoded High Resolution frame and the upscaled Low Resolutionone. If a given macro-block has been correctly decoded by both the HighResolution and the Low Resolution, only the High Resolution macro-blockis used: the Low Resolution macro-block is discarded because it haslower quality.

On the other side, if a macro-block has been correctly decoded only inone of the two frames, the postprocessor 150 will use only the correctone. If a macro-block is corrupted in both the High Resolution and theLow Resolution frames, then the postprocessor 150 will copy thecorresponding macro-block from the previous concealed frame (stored inbuffer 155).

After concealment, the output frame is sent to the playback unit and tothe High Resolution decoder and the Low Resolution one (passing throughthe downscaler 130). The frame is also copied in the postprocessorbuffer 155 for future reference.

Sending the concealed frames back to the High Resolution and the LowResolution decoders is useful because it effectively limits errorpropagation. Without this feedback, in fact, the decoders would not beable to correctly decode frames depending on corrupted reference frames.

This joint decoding process results in a concrete improvement of videoquality compared to the independent decoding of the High Resolution andthe Low Resolution streams. When a concealed frame is based on the LowResolution stream, its quality is not optimal but intelligibility ispreserved.

The Unbalanced Multiple Descriptions technique is suitable formulticast/broadcast video distribution where estimating packet loss canbe a difficult task because different receivers experience differentloss patterns.

Multicast video streaming over a Wireless LAN network is a possiblescenario where this technique provides good results. This scenario isshown in FIG. 5.

In this scenario, a video streaming server 200 in a hotspot Access Point210 could generate the Unbalanced Multiple Descriptions packets in orderto increase video resilience to packet loss. The Unbalanced MultipleDescriptions packets include the High Resolution and the Low Resolutionstreams, as described in the foregoing. The WLAN Access Point 210 sendsthe multicast Unbalanced Multiple Descriptions video packets to theLaptops associated to it.

The video can be correctly decoded by Unbalanced MultipleDescriptions-aware laptop PCs 220, PDAs and handheld devices,independently of their specific packet loss pattern. No feedback isrequired from the receivers. Each laptop 220 is able to reconstruct thevideo even if some Unbalanced Multiple Descriptions packets get lost.

FIG. 6 shows the video streaming over a 3G scenario.

Another scenario where the solution can provide good results is thevideo broadcasting over a 3G cells (Third generation mobile cells).

In this case, a video streaming server 300 could generate UnbalancedMultiple Descriptions packets to be sent in broadcast via a 3G basestation 310. The Unbalanced Multiple Descriptions packets include theHigh Resolution and the Low Resolution streams, as described. The 3Gbase station 310 broadcasts the Unbalanced Multiple Descriptions videopackets to 3G mobile phones 320 in its cell. Each mobile phone 320 withUnbalanced Multiple Descriptions video capabilities will be able toconceal the errors caused by lost packets independently of its specificpacket loss pattern.

It is now considered the case where a small set of receivers suffersheavy packet loss (the mobile phones near the border of the cell), whilethe others (the majority) experience only occasional losses. The ForwardError Correction based techniques would require a high percentage ofredundant data in order to guarantee acceptable quality to this smallset of receivers. The Unbalanced Multiple Descriptions, instead, wouldrequire less redundancy to achieve the same goal. The same is true inthe case where receivers experience good channel conditions for themajority of the time, and suffer heavy losses only for short periods(for example during hand-off). No feedback is required from thereceivers.

The technique described herein provides good results even when there arelong bursts of losses or when loss percentage is more than redundancypercentage. In particular, the Unbalanced Multiple Descriptions iseffective even if loss percentage has been underestimated.

Consequently, without prejudice to the underlying principles of theinvention, the details and the embodiments may vary, also appreciably,with reference to what has been described by way of example only,without departing from the scope of the invention as defined by theannexed claims.

All of the above U.S. patents, U.S. patent application publications,U.S. patent applications, foreign patents, foreign patent applicationsand non-patent publications referred to in this specification and/orlisted in the Application Data Sheet are incorporated herein byreference, in their entirety.

1. A system for receiving and decoding, to macroblocks of frames, asignal comprising a single packet aggregate of a high resolution packetand a low resolution packet, said signal produced by unbalanced multipledescription coding of video signals using different intra refreshperiods, an intra refresh period for said low resolution packet beingshorter than the intra refresh period for said high resolution packet,said system comprising: a de-packetizer module configured for extractingsaid high resolution packet and said low resolution packet from saidsingle packet, said de-packetizer module sending said extracted highresolution packet and said low resolution packet to a respective firstdecoder and a second decoder, said first decoder and said second decodersupplying respectively a high resolution and a low resolution frame to apostprocessor, said postprocessor configured for performing an errorconcealment operation.
 2. The system of claim 1 wherein thepostprocessor is further configured for recovering from a low resolutionframe macro-block when a given a high resolution frame macro-block hasbeen lost.
 3. The system of claim 1 wherein the postprocessor is furtherconfigured for performing error concealment operations by operating onsaid high resolution frames and said low resolution frames with a samesequence number, producing a playback frame that is sent to a playbackunit, said playback frame being also sent back to said first decoder andsaid second decoder.
 4. The system of claim 1 further comprising anupscaler module configured for receiving said low resolution frames fromsaid low second decoder, said upscaler further configured for upscalingsaid low resolution frames for input to the postprocessor.